Three-dimensional (3D) type sensing systems are commonly used to generate 3D images of a location for use in various applications. For example, such 3D images are used for creating a safe training environment for military operations or civilian activities; for generating topographical maps; or for surveillance of a location. Such sensing systems typically operate by capturing elevation data associated with the location of the target. One example of a 3D type sensing system is a Light Detection and Ranging (LIDAR) system. The LIDAR type 3D sensing systems generate data by recording multiple range echoes from a single pulse of laser light to generate a frame, sometimes referred to as an image frame. Accordingly, each image frame of LIDAR data includes a collection of points in three dimensions (3D point cloud), which correspond to multiple range echoes within a sensor's aperture. These points can be organized into “voxels” which represent values on a regular grid in a three dimensional space. Voxels used in 3D imaging are analogous to pixels used in the context of 2D imaging devices. These frames can be processed to reconstruct a 3D image of the location of the target. In this regard, each point in the 3D point cloud has an individual x, y and z value, representing the actual surface within the scene in 3D.
A three dimensional (3D) point cloud is a dataset composed of spatial measurement of positions in 3D space (x, y, z), where x and y are cross-range spatial positions and z is height. The 3D data is generated by systems capable of scanning surfaces, such as stereo paired cameras, radars, LIDAR sensors, etc. Point cloud visualization, in general, is of great interest within the defense and geospatial community. Image and geospatial analysts, however, have difficulty using point cloud data alongside traditional 2D imagery. In many situations, point cloud datasets are viewed by coloring height, or altitude (z) information based on a single color composition.
One color composition may be based on a hue, saturation, and intensity (HSI) color space. Hue is the color, saturation is the color contrast, and intensity is the brightness. An HSI color space for coloring point clouds is disclosed in U.S. Patent Publication No. 2009/0231327, published on Sep. 17, 2009, which is incorporated herein by reference in its entirety. That application is titled: “Method For Visualization of Point Cloud Data”.
Another U.S. Patent Application, Publication No. 2010/0208981, which is titled “Method For Visualization of Point Cloud Data Based on Scene Content”, published on Aug. 19, 2010, includes a description of using radiometric information of a 2D image for coloring point clouds. This application is incorporated herein by reference in its entirety. In this application, radiometric information is obtained from 2D imagery and used to color a point cloud in 3D space. Although the application states that the coloring schemes may be applied to multiple frames of data, there is no disclosure of using multi-color compositions in real-time for 4D point cloud applications.
Thus, color maps have been used to help visualize point cloud data. For example, a color map may be used to selectively vary color of each point in a 3D point cloud in accordance with a predefined variable, such as altitude. In such systems, variations in color are used to signify points at different heights, or altitudes above ground level. No methodology exists, however, for coloring a time sequence of point clouds in a real-time environment. No methodology exists for using multi-color compositions in real-time for object hazard avoidance, without having prior position information.
Advances in LIDAR systems have been pushing towards 4D data (x, y, z and time, t). These systems are capable of operating in the same way as a video camera operates, at 30 frames per second. Sampling a scene in a 4D domain is very attractive in military and civilian applications. As will be explained, the present invention uses 4D measurements recorded by a LIDAR system to generate 3D video. The present invention visualized the data for hazard avoidance, as well as 3D information. In order to have both functionalities (hazard avoidance and 3D information) at the same time, two separate color compositions are used. One color composition is used for coloring a scene based on height, while the second color composition is layered on top of the first color composition for coloring points that represent a hazard or a vertical obstruction to the viewer.